System and method for read synchronization of memory modules

ABSTRACT

A memory module includes several memory devices coupled to a memory hub. The memory hub includes several link interfaces coupled to respective processors, several memory controller coupled to respective memory devices, a cross-bar switch coupling any of the link interfaces to any of the memory controllers, a write buffer and read cache for each memory device and a read synchronization module. The read synchronization module includes a write pointer, a read pointer and a buffer. The write pointer is incremented in response to the receipt of read data. The read pointer increments in response to coupling of the read data from the memory hub. A comparator compares the read pointer an the write pointer, and the comparison is used to adjust the memory timing.

TECHNICAL FIELD

The present invention relates to a processor-based system, and moreparticularly, to a processor-based system having a memory module with amemory hub coupling several memory devices to a processor or othermemory access devices.

BACKGROUND OF THE INVENTION

Processor-based systems, such as computer systems, use memory devices,such as dynamic random access memory (“DRAM”) devices, to storeinstructions and data that are accessed by a processor. These memorydevices are typically used as system memory in a computer system. In atypical computer system, the processor communicates with the systemmemory through a processor bus and a memory controller. The processorissues a memory request, which includes a memory command, such as a readcommand, and an address designating the location from which data orinstructions are to be read. The memory controller uses the command andaddress to generate appropriate command signals as well as row andcolumn addresses, which are applied to the system memory. In response tothe commands and addresses, data is transferred between the systemmemory and the processor. The memory controller is often part of asystem controller, which also includes bus bridge circuitry for couplingthe processor bus to an expansion bus, such as a PCI bus.

Although the operating speed of memory devices has continuouslyincreased, this increase in operating speed has not kept pace withincreases in the operating speed of processors. Even slower has been theincrease in operating speed of memory controllers coupling processors tomemory devices. The relatively slow speed of memory controllers andmemory devices limits the data bandwidth between the processor and thememory devices.

In addition to the limited bandwidth between processors and memorydevices, the performance of computer systems is also limited by latencyproblems that increase the time required to read data from system memorydevices. More specifically, when a memory device read command is coupledto a system memory device, such as a synchronous DRAM (“SDRAM”) device,the read data are output from the SDRAM device only after a delay ofseveral clock periods. Therefore, although SDRAM devices cansynchronously output burst data at a high data rate, the delay ininitially providing the data can significantly slow the operating speedof a computer system using such SDRAM devices.

One approach to alleviating the memory latency problem is to usemultiple memory devices coupled to the processor through a memory hub.In a memory hub architecture, a system controller or memory hubcontroller is coupled to several memory modules, each of which includesa memory hub coupled to several memory devices. The memory hubefficiently routes memory requests and responses between the controllerand the memory devices. Computer systems employing this architecture canhave a higher bandwidth because a processor can access one memory modulewhile another memory module is responding to a prior memory access. Forexample, the processor can output write data to one of the memorymodules in the system while another memory module in the system ispreparing to provide read data to the processor. The operatingefficiency of computer systems using a memory hub architecture can makeit more practical to vastly increase data bandwidth of a memory system.A memory hub architecture can also provide greatly increased memorycapacity in computer systems.

Although there are advantages to utilizing a memory hub for accessingmemory devices, the design of the hub memory system, and more generally,computer systems including such a memory hub architecture, becomesincreasingly difficult. For example, in many hub based memory systems,the processor is coupled through a memory hub controller to each ofseveral memory hubs via a high speed bus or link over which signals,such as command, address, or data signals, are transferred at a veryhigh rate. The memory hubs are, in turn, coupled to several memorydevices via buses that must also operate at a very high speed. However,as transfer rates increase, the time for which a signal represents validinformation is decreasing. As commonly referenced by those ordinarilyskilled in the art, the window or “eye” for when the signals are validdecreases at higher transfer rates. With specific reference to datasignals, the “data eye” decreases. As understood by one skilled in theart, the data eye for each of the data signals defines the actualduration that each signal is valid after various factors affecting thesignal are considered, such as timing skew, voltage and current drivecapability, and the like. In the case of timing skew of signals, itoften arises from a variety of timing errors such as loading on thelines of the bus and the physical lengths of such lines.

As data eyes of signals decrease at higher transfer rates, it ispossible that one or more of a groups of signals provided by a memorydevice in parallel will have different arrival times at a memory hub towhich the memory devices are coupled. As a result, not all of thesignals will be simultaneously valid at the memory hub, thus preventingthe memory hub from successfully capturing the signals. For example,where a plurality of signals are provided in parallel over a bus, thedata eye of one or more of the particular signals do not overlap withthe data eyes of the other signals. In this situation, the signalshaving non-overlapping data eyes are not valid at the same time as therest of the signals, and consequently, cannot be successfully capturedby the memory hub. Clearly, as those ordinarily skilled in the art willrecognize, the previously described situation is unacceptable.

One approach to alleviating timing problems in memory devices is to usea delay-locked loop (DLL) or delay line (DL) to lock or align thereceipt of read data from a memory device to a capture strobe signalused to latch the read data in a memory hub. More specifically, a readstrobe signal is output by the memory devices along with read datasignals. At higher transfer rates, the timing of the read strobe signalcan vary so that it cannot be reliably used to capture the read datasignals in the memory hub. Further, even if the read data strobe couldreliably capture the read data signals in the memory hub, the time atwhich the read data signals were captured could vary in relation to acore clock domain used to control the operation of the memory hub thatis coupled to the memory device. In such case, the read data may not bepresent in the memory hub at the proper time. To alleviate this problem,the timing of the read data strobe signals is adjusted using the DLL orDL to generate a capture clock signal that can reliably capture the readdata signals. The DLL or DL is thus effective in preventing substantialdrifting of a read data eye in relation to the core clock domain. Astransfer rates increase, however, the timing specifications for the DLLor DL become more stringent and therefore increasingly difficult tomeet. Furthermore, the amount of circuitry required to implement asuitable DLL or DL can materially reduce the amount of space that couldotherwise be used for memory device circuitry, thereby either increasingthe cost or reducing the storage capacity of such memory devices.

There is accordingly a need for a system and method that avoids the needto precisely control the timing relationships between a memory hub clockdomain and the receipt of read data signals at the memory hub in amanner that avoids the need for extensive DLL or DL circuitry.

SUMMARY OF THE INVENTION

A memory module for a processor-based system includes a plurality ofmemory devices coupled to a memory hub. The memory hub includes a linkinterface for receiving memory requests for access to the memory devicesand at least one memory device interface coupled to the memory devices.The memory device interface couples write memory requests and write datato the memory devices, and couples read memory requests to the memorydevice and read data from the memory device. The memory hub alsoincludes a read synchronization module coupled to the memory deviceinterface. The read synchronization module is operable to compare timingbetween the received read data and the memory requests. The memory hubfurther includes a memory sequencer coupled to the link interface andthe memory device interface. The memory sequencer is operable to couplememory requests to the memory device interface responsive to memoryrequests received from the link interface, the memory sequencer furtherbeing operable to dynamically adjust operability responsive to the readsynchronization module comparison.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a computer system according to one exampleof the invention in which a memory hub is included in each of aplurality of memory modules.

FIG. 2 is a block diagram of a memory hub used in the computer system ofFIG. 1, which contains read synchronization modules according to oneexample of the invention.

FIG. 3 is a block diagram of one embodiment of a synchronization systemaccording to one example of the invention.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the present invention are directed to a memory hub modulehaving the capability to perform a read channel synchronization. Certaindetails are set forth below to provide a sufficient understanding ofvarious embodiments of the invention. However, it will be clear to oneskilled in the art that the invention may be practiced without theseparticular details. In other instances, well-known circuits, controlsignals, and timing protocols have not been shown in detail in order toavoid unnecessarily obscuring the invention.

A computer system 100 according to one example of the invention is shownin FIG. 1. The computer system 100 includes a processor 104 forperforming various computing functions, such as executing specificsoftware to perform specific calculations or tasks. The processor 104includes a processor bus 106 that normally includes an address bus, acontrol bus, and a data bus. The processor bus 106 is typically coupledto cache memory 108, which, as previously mentioned, is usually staticrandom access memory (“SRAM”). Finally, the processor bus 106 is coupledto a system controller 110, which is also sometimes referred to as a“North Bridge” or “memory controller.”

The system controller 110 serves as a communications path to theprocessor 104 for a variety of other components. More specifically, thesystem controller 110 includes a graphics port that is typically coupledto a graphics controller 112, which is, in turn, coupled to a videoterminal 114. The system controller 110 is also coupled to one or moreinput devices 118, such as a keyboard or a mouse, to allow an operatorto interface with the computer system 100. Typically, the computersystem 100 also includes one or more output devices 120, such as aprinter, coupled to the processor 104 through the system controller 110.One or more data storage devices 124 are also typically coupled to theprocessor 104 through the system controller 110 to allow the processor104 to store data or retrieve data from internal or external storagemedia (not shown). Examples of typical storage devices 124 include hardand floppy disks, tape cassettes, and compact disk read-only memories(CD-ROMs).

The system controller 110 is coupled to several memory modules 130 a,b .. . n, which serve as system memory for the computer system 100. Thememory modules 130 are preferably coupled to the system controller 110through respective high-speed links 134 a and 134 b, which may beoptical or electrical communication paths or some other type ofcommunications paths. The high speed link 134 a is the downlink,carrying memory requests from the memory hub controller 132 to thememory modules 130 a-n. The high speed link 134 b is the uplink,carrying memory responses from the memory modules 130 a-n to the memoryhub controller 132. In the event the high-speed links 134 a and 134 bare implemented as optical communication paths, the opticalcommunication paths may be in the form of one or more optical fibers,for example. In such case, the system controller 110 and the memorymodules will include an optical input/output port or separate input andoutput ports coupled to the optical communication paths. The memorymodules 130 are shown coupled to the system controller 110 in amulti-drop arrangement in which the high-speed links 134 a and 134 b arecoupled to all of the memory modules 130. However, it will be understoodthat other topologies may also be used, such as a point-to-pointcoupling arrangement in which a separate high-speed link (not shown) isused to couple each of the memory modules 130 to the system controller110. A switching topology may also be used in which the systemcontroller 110 is selectively coupled to each of the memory modules 130through a switch (not shown). Other topologies that may be used will beapparent to one skilled in the art.

Each of the memory modules 130 includes a memory hub 140 for controllingaccess to 32 memory devices 148, which, in the example illustrated inFIG. 1, are synchronous dynamic random access memory (“SDRAM”) devices.However, a fewer or greater number of memory devices 148 may be used,and memory devices other than SDRAM devices may, of course, also beused. In the example illustrated in FIG. 1, the memory hubs 140communicate over 4 independent memory channels 149 over the high-speedlinks 134 a and 134 b. In this example, although not shown in FIG. 1, 4memory hub controllers 128 are provided, each to receive data from onememory channel 149. A fewer or greater number of memory channels 149 maybe used, however, in other examples. The memory hub 140 is coupled toeach of the system memory devices 148 through a bus system 150, whichnormally includes a control bus, an address bus and a data bus.

A memory hub 200 according to an embodiment of the present invention isshown in FIG. 2. The memory hub 200 can be substituted for the memoryhub 140 of FIG. 1. The memory hub 200 is shown in FIG. 2 as beingcoupled to four memory devices 240 a-d, which, in the present exampleare conventional SDRAM devices. In an alternative embodiment, the memoryhub 200 is coupled to four different banks of memory devices, ratherthan merely four different memory devices 240 a-d, with each banktypically having a plurality of memory devices. However, for the purposeof providing an example, the present description will be with referenceto the memory hub 200 coupled to the four memory devices 240 a-d. Itwill be appreciated that the necessary modifications to the memory hub200 to accommodate multiple banks of memory is within the knowledge ofthose ordinarily skilled in the art.

Further included in the memory hub 200 are link interfaces 210 a-d and212 a-d for coupling the memory module on which the memory hub 200 islocated to a first high speed data link 220 and a second high speed datalink 222, respectively. As previously discussed with respect to FIG. 1,the high-speed data links 220, 222 can be implemented using an opticalor electrical communication path or some other type of communicationpath. The link interfaces 210 a-d, 212 a-d are conventional, and includecircuitry used for transferring data, command, and address informationto and from the high speed data links 220, 222. As well known, suchcircuitry includes transmitter and receiver logic known in the art. Itwill be appreciated that those ordinarily skilled in the art havesufficient understanding to modify the link interfaces 210 a-d, 212 a-dto be used with specific types of communication paths, and that suchmodifications to the link interfaces 210 a-d, 212 a-d can be madewithout departing from the scope of the present invention. For example,in the event the high-speed data link 220, 222 is implemented using anoptical communications path, the link interfaces 210 a-d, 212 a-d willinclude an optical input/output port that can convert optical signalscoupled through the optical communications path into electrical signals.

The link interfaces 210 a-d, 212 a-d are coupled to a switch 260 througha plurality of bus and signal lines, represented by busses 214. Thebusses 214 are conventional, and include a write data bus and a readdata bus, although a single bi-directional data bus may alternatively beprovided to couple data in both directions through the link interfaces210 a-d, 212 a-d. It will be appreciated by those ordinarily skilled inthe art that the busses 214 are provided by way of example, and that thebusses 214 may include fewer or greater signal lines, such as furtherincluding a request line and a snoop line, which can be used formaintaining cache coherency.

The link interfaces 210 a-d, 212 a-d include circuitry that allow thememory hub 200 to be connected in the system memory in a variety ofconfigurations. For example, the multi-drop arrangement, as shown inFIG. 1, can be implemented by coupling each memory module to the memoryhub controller 128 through either the link interfaces 210 a-d or 212a-d. Alternatively, a point-to-point, or daisy chain configuration canbe implemented by coupling the memory modules in series. For example,the link interfaces 210 a-d can be used to couple a first memory moduleand the link interfaces 212 a-d can be used to couple a second memorymodule. The memory module coupled to a processor, or system controller,will be coupled thereto through one set of the link interfaces andfurther coupled to another memory module through the other set of linkinterfaces. In one embodiment of the present invention, the memory hub200 of a memory module is coupled to the processor in a point-to-pointarrangement in which there are no other devices coupled to theconnection between the processor 104 and the memory hub 200. This typeof interconnection provides better signal coupling between the processor104 and the memory hub 200 for several reasons, including relatively lowcapacitance, relatively few line discontinuities to reflect signals andrelatively short signal paths.

The switch 260 is further coupled to four memory interfaces 270 a-dwhich are, in turn, coupled to the system memory devices 240 a-d,respectively. By providing a separate and independent memory interface270 a-d for each system memory device 240 a-d, respectively, the memoryhub 200 avoids bus or memory bank conflicts that typically occur withsingle channel memory architectures. The switch 260 is coupled to eachmemory interface through a plurality of bus and signal lines,represented by busses 274. The busses 274 include a write data bus, aread data bus, and a request line. However, it will be understood that asingle bi-directional data bus may alternatively be used instead of aseparate write data bus and read data bus. Moreover, the busses 274 caninclude a greater or lesser number of signal lines than those previouslydescribed.

In an embodiment of the present invention, each memory interface 270 a-dis specially adapted to the system memory devices 240 a-d to which it iscoupled. More specifically, each memory interface 270 a-d is speciallyadapted to provide and receive the specific signals received andgenerated, respectively, by the system memory device 240 a-d to which itis coupled. Also, the memory interfaces 270 a-d are capable of operatingwith system memory devices 240 a-d operating at different clockfrequencies. As a result, the memory interfaces 270 a-d isolate theprocessor 104 from changes that may occur at the interface between thememory hub 230 and memory devices 240 a-d coupled to the memory hub 200,and it provides a more controlled environment to which the memorydevices 240 a-d may interface.

The switch 260 coupling the link interfaces 210 a-d, 212 a-d and thememory interfaces 270 a-d can be any of a variety of conventional orhereinafter developed switches. For example, the switch 260 may be across-bar switch that can simultaneously couple link interfaces 210 a-d,212 a-d and the memory interfaces 270 a-d to each other in a variety ofarrangements. The switch 260 can also be a set of multiplexers that donot provide the same level of connectivity as a cross-bar switch butnevertheless can couple the some or all of the link interfaces 210 a-d,212 a-d to each of the memory interfaces 270 a-d. The switch 260 mayalso includes arbitration logic (not shown) to determine which memoryaccesses should receive priority over other memory accesses. Busarbitration performing this function is well known to one skilled in theart.

With further reference to FIG. 2, each of the memory interfaces 270 a-dincludes a respective memory controller 280, a respective write buffer282, and a respective cache memory unit 284. The memory controller 280performs the same functions as a conventional memory controller byproviding control, address and data signals to the system memory device240 a-d to which it is coupled and receiving data signals from thesystem memory device 240 a-d to which it is coupled. The write buffer282 and the cache memory unit 284 include the normal components of abuffer and cache memory, including a tag memory, a data memory, acomparator, and the like, as is well known in the art. The memorydevices used in the write buffer 282 and the cache memory unit 284 maybe either DRAM devices, static random access memory (“SRAM”) devices,other types of memory devices, or a combination of all three.Furthermore, any or all of these memory devices as well as the othercomponents used in the cache memory unit 284 may be either embedded orstand-alone devices.

The write buffer 282 in each memory interface 270 a-d is used to storewrite requests while a read request is being serviced. In such a system,the processor 104 can issue a write request to a system memory device240 a-d even if the memory device to which the write request is directedis busy servicing a prior write or read request. Using this approach,memory requests can be serviced out of order since an earlier writerequest can be stored in the write buffer 282 while a subsequent readrequest is being serviced. The ability to buffer write requests to allowa read request to be serviced can greatly reduce memory read latencysince read requests can be given first priority regardless of theirchronological order. For example, a series of write requestsinterspersed with read requests can be stored in the write buffer 282 toallow the read requests to be serviced in a pipelined manner followed byservicing the stored write requests in a pipelined manner. As a result,lengthy settling times between coupling write request to the memorydevices 270 a-d and subsequently coupling read request to the memorydevices 270 a-d for alternating write and read requests can be avoided.

The use of the cache memory unit 284 in each memory interface 270 a-dallows the processor 104 to receive data responsive to a read commanddirected to a respective system memory device 240 a-d without waitingfor the memory device 240 a-d to provide such data in the event that thedata was recently read from or written to that memory device 240 a-d.The cache memory unit 284 thus reduces the read latency of the systemmemory devices 240 a-d to maximize the memory bandwidth of the computersystem. Similarly, the processor 104 can store write data in the cachememory unit 284 and then perform other functions while the memorycontroller 280 in the same memory interface 270 a-d transfers the writedata from the cache memory unit 284 to the system memory device 240 a-dto which it is coupled.

Further included in the memory hub 200 is a built in self-test (BIST)and diagnostic engine 290 coupled to the switch 260 through a diagnosticbus 292. The diagnostic engine 290 is further coupled to a maintenancebus 296, such as a System Management Bus (SMBus) or a maintenance busaccording to the Joint Test Action Group (JTAG) and IEEE 1149.1standards. Both the SMBus and JTAG standards are well known by thoseordinarily skilled in the art. Generally, the maintenance bus 296provides a user access to the diagnostic engine 290 in order to performmemory channel and link diagnostics. For example, the user can couple aseparate PC host via the maintenance bus 296 to conduct diagnostictesting or monitor memory system operation. By using the maintenance bus296 to access diagnostic test results, issues related to the use of testprobes, as previously discussed, can be avoided. It will be appreciatedthat the maintenance bus 296 can be modified from conventional busstandards without departing from the scope of the present invention. Itwill be further appreciated that the diagnostic engine 290 shouldaccommodate the standards of the maintenance bus 296, where such astandard maintenance bus is employed. For example, the diagnostic engineshould have an maintenance bus interface compliant with the JTAG busstandard where such a maintenance bus is used.

Further included in the memory hub 200 is a DMA engine 286 coupled tothe switch 260 through a bus 288. The DMA engine 286 enables the memoryhub 200 to move blocks of data from one location in the system memory toanother location in the system memory without intervention from theprocessor 104. The bus 288 includes a plurality of conventional buslines and signal lines, such as address, control, data busses, and thelike, for handling data transfers in the system memory. Conventional DMAoperations well known by those ordinarily skilled in the art can beimplemented by the DMA engine 286. The DMA engine 286 is able to read alink list in the system memory to execute the DMA memory operationswithout processor intervention, thus, freeing the processor 104 and thebandwidth limited system bus from executing the memory operations. TheDMA engine 286 can also include circuitry to accommodate DMA operationson multiple channels, for example, for each of the system memory devices240 a-d. Such multiple channel DMA engines are well known in the art andcan be implemented using conventional technologies.

The diagnostic engine 290 and the DMA engine 286 are preferably embeddedcircuits in the memory hub 200. However, including separate a diagnosticengine and a separate DMA device coupled to the memory hub 200 is alsowithin the scope of the present invention.

Embodiments of the present invention provide a read synchronizationmodule 297 for controlling the timing of read requests sent to thememory devices 240 so that read data signals are received at the memoryhub 200 at the proper time in relation to a system clock signal used toestablish a clock domain for the memory hub 200. Although a singlesynchronization module 297 is shown in FIG. 2, it is to be understoodthat a plurality of synchronization modules 297 may also be used, forexample, one per memory controller 280. Further, in the embodiment shownin FIG. 2, the synchronization module 297 is shown in communication withthe memory device 240 c and the memory controller 280 c. In someembodiments, the synchronization module 297 may be in communication withone or more memory devices and the controller 100 or memory hub 140shown in FIG. 1. As mentioned above, the memory synchronization module297 functions to synchronize the coupling of read data from the memorydevice with the core clock domain of the memory hub 200 as establishedby a system clock signal from the memory hub controller 128.Accordingly, if data is sent by the memory devices 148 either too earlyor too late, the read data might be coupled to the memory hub 200 at atime that is not synchronized to the core clock domain of the memory hub200. Significantly, the synchronization module 297 allows the timing ofa strobe signal used to capture read data signals to drift as needed sothat the read data signals are captured at the proper time in relationto the core clock domain.

FIG. 3 illustrates a read synchronization module 300 according to anembodiment of the present invention that can be used as the readsynchronization module 297 shown in FIG. 2. It will be appreciated thatFIG. 3 is a functional block diagram representative of a suitablesynchronization module and is not intended to limit the scope of thepresent invention. The functional blocks shown in FIG. 3 areconventional, and can be implemented using well known techniques andcircuitry. It will be further appreciated that control signals and otherfunctional blocks have been omitted from FIG. 3 in order to avoidunnecessarily obscuring the present invention, and that the descriptionprovided herein is sufficient to enable those ordinarily skilled in theart to practice the invention.

Included in the read synchronization module 300 is a memory sequencer304 that generates properly timed signals for controlling the operationof the memory devices 148 (FIG. 1) or 240 (FIG. 2). However, inalternative embodiments, the DMA engine 286 may be used for thispurpose. The nature of the signals generated by the memory sequencer 304will, of course, be determined by the nature of the signals used by thememory devices 148, 240. The timing of the signals controlling theoperation of the memory devices 148, 240 control the time when read datasignals are output from the memory devices 148, 240.

A buffer 308 is used to store read data received from one or more of thememory devices 148, 240. The buffer 308 in FIG. 3 is a first-infirst-out (FIFO) buffer, such as a circular buffer, and may beimplemented as known in the art. The buffer 308 is clocked with a readstrobe signal, which may also be referred to as a read clock signal. Theread strobe signal is generated by the memory devices 148, 240 and isoutput from the memory devices 148, 240 along with read data signals.When the read data is clocked into the buffer 308 by the read strobesignal, i.e., the read data are written to the buffer 308, a writepointer, 312 is incremented. The read data are clocked out of the buffer308 and coupled to the memory hub controller 132 (FIG. 1) by a coreclock signal, which may be derived from a system clock signal. When datais clocked out of the buffer 308 by the core clock, i.e., the read dataare read from the buffer 308, a read pointer 314 is incremented. Theread pointer 314 and the write pointer 312 are then compared by acomparator 316. Comparator 316 generates an adjust signal in response tothe comparison. Generally, the relationship between the read pointer 314and the write pointer 312 identifies the crossing margin from the memorydevice timing domain represented by the read strobe signal to the coreclock timing domain—the “data eye”, as described above.

The adjust signal is fed back to the memory sequencer 304. The data eyewill decrease, i.e., the read pointer 314 will be too close to the writepointer 312, if the read data are being coupled from the memory devices148, 240 too early in relation to the core clock coupling the read datato the memory hub controller 128. In such case, the memory sequencer 304reduces the rate at which read data are coupled from the memory devices148. Conversely, the data eye will increase, i.e., the read pointer 314will be too far away from the write pointer 312, if the read data arebeing coupled from the memory devices 148 too late in relation to thecore clock coupling the read data to the memory hub controller 128. Insuch case, the memory sequencer 304 increases the rate at which readdata are coupled from the memory devices 148. As a result, the read dataare coupled from the memory devices 148 at a rate that is adjusted tomatch the timing of the core clock signal.

From the foregoing it will be appreciated that, although specificembodiments of the invention have been described herein for purposes ofillustration, various modifications may be made without deviating fromthe spirit and scope of the invention. Accordingly, the invention is notlimited except as by the appended claims.

1. A memory module, comprising: a plurality of memory devices; and amemory hub, comprising: a link interface receiving memory requests foraccess to at least one of the memory devices; a memory device interfacecoupled to the memory devices, the memory device interface beingoperable to couple memory requests to the memory devices for access toat least one of the memory devices and to receive read data responsiveto at least some of the memory requests; a read synchronization modulecoupled to the memory device interface, the read synchronization moduleoperable to compare timing between coupling read data from the memorydevices and coupling read data from the memory hub and to generate anadjust signal corresponding to the compared timing; and a memorysequencer coupled to the link interface, the memory device interface,and the read synchronization module, the memory sequencer being operableto couple memory requests to the memory device interface responsive tomemory requests received from the link interface, the memory sequencerfurther being operable to adjust the timing at which read memoryrequests are coupled to the memory device interface responsive to theadjust signal.
 2. The memory module of claim 1 wherein the linkinterface comprises an optical input/output port.
 3. The memory moduleof claim 1 wherein the read synchronization module comprises a buffercoupled to at least one memory device and the memory sequencer, thebuffer operable to store received read data and to subsequently outputthe stored read data.
 4. The memory module of claim 3 wherein the readsynchronization module comprises: a write pointer coupled to at leastone memory device, the write pointer operable to increment in responseto read data being stored in the buffer; a read pointer coupled to thememory sequencer, the read pointer operable to increment in response toread data being output from the buffer; and a comparator coupled to theread pointer, the write pointer, and the memory sequencer, thecomparator operable to compare the read pointer to the write pointer andto generate the adjust signal corresponding to the compared timing. 5.The memory module of claim 1 wherein the read synchronization modulecomprises: a write pointer coupled to at least one memory device, thewrite pointer operable to increment in response to a read strobe coupledto the memory device; a read pointer coupled to the memory sequencer,the read pointer operable to increment in response to read data beingoutput from the memory module; and a comparator coupled to the readpointer, the write pointer, and the memory sequencer, the comparatoroperable to compare the read pointer to the write pointer and togenerate the adjust signal corresponding to the compared timing.
 6. Thememory module of claim 1 wherein the memory devices comprise dynamicrandom access memory devices.
 7. The memory module of claim 1 whereinthe memory sequencer is operable to increase a period between receivingthe memory requests from the link interface and coupling the memoryrequests to the memory device interface responsive to an adjust signalindicative of a decrease in the timing between coupling read data fromthe memory devices and coupling read data from the memory hub, thememory sequencer further being operable to decrease the period betweenreceiving the memory requests from the link interface and coupling thememory requests to the memory device interface responsive to an adjustsignal indicative of an increase in the timing between coupling readdata from the memory devices and coupling read data from the memory hub.8. A memory module, comprising: a plurality of memory devices, each ofthe memory devices being operable to output read data signals and a readdata strobe signal responsive to respective memory requests; and amemory hub, comprising: a link interface receiving memory requests foraccess to at least one of the memory devices; a memory device interfacecoupled to the memory devices, the memory device interface beingoperable to couple the received memory requests to at least one of thememory devices and to receive the read data signals and the read datastrobe signal responsive to respective memory requests; a buffer coupledto receive the read data signals, the read data signals being clockedinto the buffer responsive to the read data strobe signal; a readsynchronization module coupled to the memory device interface, the readsynchronization module operable to compare timing between the read datastrobe signals and a core clock signal and to generate an adjust signalcorresponding to the compared timing; and a memory sequencer coupled tothe link interface, the memory device interface, and the readsynchronization module, the memory sequencer being operable to couplememory requests to the memory device interface responsive to memoryrequests received from the link interface, the memory sequencer furtherbeing operable to adjust the timing at which read memory requests arecoupled to the memory device interface responsive to the adjust signal.9. The memory module of claim 8 wherein the link interface comprises anoptical input/output port.
 10. The memory module of claim 8 wherein theread data signals are clocked out of the buffer responsive to the coreclock signal.
 11. The memory module of claim 10 wherein the readsynchronization module comprises: a write pointer coupled to at leastone memory device, the write pointer operable to increment in responseto the read data strobe; a read pointer coupled to the memory sequencer,the read pointer operable to increment in response to the core clocksignal; and a comparator coupled to the read pointer, the write pointer,and the memory sequencer, the comparator operable to compare the readpointer to the write pointer and to generate the adjust signalcorresponding to the compared timing.
 12. The memory module of claim 8wherein the memory devices comprise dynamic random access memorydevices.
 13. A memory hub, comprising: a link interface receiving memoryrequests for access to memory cells in at least one memory device; amemory device interface coupled to a plurality of memory devices, thememory device interface being operable to couple memory requests to thememory devices for access to at least one of the memory devices and toreceive read data responsive to at least some of the memory requests; aread synchronization module coupled to the memory device interface, theread synchronization module operable to compare timing between couplingread data from the memory devices and coupling read data from the memoryhub and to generate an adjust signal corresponding to the comparedtiming; and a memory sequencer coupled to the link interface, the memorydevice interface, and the read synchronization module, the memorysequencer being operable to couple memory requests to the memory deviceinterface responsive to memory requests received from the linkinterface, the memory sequencer further being operable to adjust thetiming at which read memory requests are coupled to the memory deviceinterface responsive to the adjust signal.
 14. The memory hub of claim13 wherein the link interface comprises an optical input/output port.15. The memory hub of claim 13 wherein the read synchronization modulecomprises a buffer coupled to at least one memory device and the memorysequencer, the buffer operable to store received read data and tosubsequently output the stored read data.
 16. The memory hub of claim 15wherein the read synchronization module comprises: a write pointercoupled to at least one memory device, the write pointer operable toincrement in response to read data being stored in the buffer; a readpointer coupled to the memory sequencer, the read pointer operable toincrement in response to read data being output from the buffer; and acomparator coupled to the read pointer, the write pointer, and thememory sequencer, the comparator operable to compare the read pointer tothe write pointer and to generate the adjust signal corresponding to thecompared timing.
 17. The memory hub of claim 13 wherein the readsynchronization module comprises: a write pointer coupled to at leastone memory device, the write pointer operable to increment in responseto a read strobe coupled to the memory device; a read pointer coupled tothe memory sequencer, the read pointer operable to increment in responseto read data being output from the memory hub; and a comparator coupledto the read pointer, the write pointer, and the memory sequencer, thecomparator operable to compare the read pointer to the write pointer andto generate the adjust signal corresponding to the compared timing. 18.The memory hub of claim 13 wherein the memory devices comprise dynamicrandom access memory devices.
 19. The memory hub of claim 13 wherein thememory sequencer is operable to increase a period between receiving thememory requests from the link interface and coupling the memory requeststo the memory device interface responsive to an adjust signal indicativeof a decrease in a period between the read pointer and the writepointer, the memory sequencer further being operable to decrease theperiod between receiving the memory requests from the link interface andcoupling the memory requests to the memory device interface responsiveto an adjust signal indicative of an increase in the period between theread pointer and the write pointer.
 20. A computer system, comprising: acentral processing unit (“CPU”); a system controller coupled to the CPU,the system controller having an input port and an output port; an inputdevice coupled to the CPU through the system controller; an outputdevice coupled to the CPU through the system controller; a storagedevice coupled to the CPU through the system controller; a plurality ofmemory modules, each of the memory modules comprising: a plurality ofmemory devices; and a memory hub, comprising: a link interface receivingmemory requests for access to at least one of the memory devices; amemory device interface coupled to the memory devices, the memory deviceinterface being operable to couple memory requests to the memory devicesfor access to at least one of the memory devices and to receive readdata responsive to at least some of the memory requests; a readsynchronization module coupled to the memory device interface, the readsynchronization module operable to compare timing between coupling readdata from the memory devices and coupling read data from the memory huband to generate an adjust signal corresponding to the compared timing;and a memory sequencer coupled to the link interface, the memory deviceinterface, and the read synchronization module, the memory sequencerbeing operable to couple memory requests to the memory device interfaceresponsive to memory requests received from the link interface, thememory sequencer further being operable to adjust the timing at whichread memory requests are coupled to the memory device interfaceresponsive to the adjust signal.
 21. The computer system of claim 20wherein the link interface comprises an optical input/output port. 22.The computer system of claim 20 wherein the read synchronization modulecomprises a buffer coupled to at least one memory device and the memorysequencer, the buffer operable to store received read data and tosubsequently output the stored read data.
 23. The computer system ofclaim 22 wherein the read synchronization module further comprises: awrite pointer coupled to at least one memory device, the write pointeroperable to increment in response to read data being stored in thebuffer; a read pointer coupled to the memory sequencer, the read pointeroperable to increment in response to read data being output from thebuffer; and a comparator coupled to the read pointer, the write pointer,and the memory sequencer, the comparator operable to compare the readpointer to the write pointer and to generate the adjust signalcorresponding to the compared timing.
 24. The computer system of claim20 wherein the read synchronization module further comprises: a writepointer coupled to at least one memory device, the write pointeroperable to increment in response to a read strobe coupled to the memorydevice; a read pointer coupled to the memory sequencer, the read pointeroperable to increment in response to read data being output from thememory module; and a comparator coupled to the read pointer, the writepointer, and the memory sequencer, the comparator operable to comparethe read pointer to the write pointer and to generate the adjust signalcorresponding to the compared timing.
 25. The computer system of claim20 wherein the memory devices comprise dynamic random access memorydevices.
 26. The computer system of claim 20 wherein the memorysequencer is operable to increase a period between receiving the memoryrequests from the link interface and coupling the memory requests to thememory device interface responsive to an adjust signal indicative of adecrease in the timing between coupling read data from the memorydevices and coupling read data from the memory hub, the memory sequencerfurther being operable to decrease the period between receiving thememory requests from the link interface and coupling the memory requeststo the memory device interface responsive to an adjust signal indicativeof an increase in the timing between coupling read data from the memorydevices and coupling read data from the memory hub.
 27. A method ofreading data from a memory module, comprising: receiving memory requestsfor access to a memory device in the memory module; coupling the memoryrequests to the memory device responsive to the received memory request,at least some of the memory requests being memory requests to read data;receiving read data responsive to the read memory requests; outputtingthe read data from the memory module; comparing timing between receivingthe read data and outputting the read from the memory module; andadjusting the timing at which read memory requests are coupled to thememory device interface as a function of the compared timing.
 28. Themethod of claim 27 further comprising storing the received read data ina buffer.
 29. The method of claim 27 wherein the buffer comprises acircular buffer.
 30. The method of claim 27 wherein the act of comparingthe timing between receiving the read data and outputting the read fromthe memory module comprises: incrementing a write pointer in response toreceiving the read data, incrementing a read pointer in response tooutputting the read data from the memory module; and comparing the writepointer to the read pointer.
 31. The method of claim 27 wherein the actof receiving memory requests for access to a memory device mounted onthe memory module comprises receiving optical signals corresponding tothe memory requests.
 32. The method of claim 27 wherein the act ofadjusting the timing at which read memory requests are coupled to thememory device interface as a function of the compared timing comprises:increasing a delay between receiving the memory requests and couplingthe memory requests to the memory device responsive to a decrease in theperiod between receiving the read data and outputting the read from thememory module, and decreasing the delay between receiving the memoryrequests and coupling the memory requests to the memory deviceresponsive to an increase in the period between receiving the read dataand outputting the read from the memory module.
 33. A method of couplingread data from a memory device to a buffer and outputting read data fromthe buffer, comprising: coupling memory requests to the memory device,at least some of the memory requests being memory requests to read data;receiving read data responsive to the read memory requests; storing thereceived read data in the buffer; outputting the read data from thebuffer; comparing timing between storing the read data in the buffer andoutputting the read from the buffer; and adjusting the timing at whichread memory requests are coupled to the memory device interface as afunction of the compared timing.
 34. The method of claim 33 wherein thebuffer comprises a circular buffer.
 35. The method of claim 33 whereinthe act of comparing the timing between storing the read data in thebuffer and outputting the read from the buffer comprises: incrementing awrite pointer in response to storing the read data in the buffer;incrementing a read pointer in response to outputting the read from thebuffer; and comparing the write pointer to the read pointer.
 36. Themethod of claim 33 wherein the act of adjusting the timing at which readmemory requests are coupled to the memory device interface comprises:increasing a delay in coupling the memory requests to the memory deviceresponsive to a decrease in the period between the write pointer and theread pointer, and decreasing the delay in coupling the memory requeststo the memory device responsive to an increase in the period between thewrite pointer and the read pointer.